Suppose you've trained a machine learning model to accomplish some task, and you'd now like to provide that model's inference capabilities as a service. Maybe you're writing an application of your own that will rely on this service, or perhaps you want to make the service available to others. This is the purpose of endpoints -- they provide a simple web-based API for feeding data to your model and getting back inference results. Azure ML currently supports three types of endpoints: batch endpoints, Kubernetes online endpoints, and managed online endpoints. I'm going to focus on batch endpoints in this post, but let me start by explaining how the three types differ. Batch endpoints are designed to handle large requests, working asynchronously and generating results that are held in blob storage.

When training a machine learning model or deploying it to an endpoint, you'll need to choose an appropriate machine to run it. I'll use the term "compute" to refer to the virtual machine (or set of machines) that runs your code in the cloud. The goal of this blog post is to give you an overview of all the compute options available to you in Azure ML, so that you can choose an appropriate option for your scenario. I'll assume that you're already familiar with the basic concepts of Azure ML, and that you have some experience using it for your own projects. Throughout this post, I'll discuss the following three major compute types available in Azure ML: I'll also briefly mention the available VM sizes, including how to get more quota for a particular VM size.